A computing/communication network is generally understood to be an interconnected or interrelated group or system of computers, peripherals, terminals, servers, switches, routers and other hardware and software based devices connected by electrical, optical or wireless transmission media to enable the transmission and reception of communications. Traditionally, telecommunication networks that communicate voice, video and data over large distances were considered to be categorically different than computer or data networks that primarily communicate digital data, typically over shorter distances. This distinction is now disappearing and emerging networking concepts and technologies are causing a convergence of voice, video, data and wireless onto, for instance, a single, multiservice network paradigm based on Internet Protocol (IP) standards. Conventionally, information transfer in networks follows a communication protocol that prescribes how digital information is encapsulated for transmission between end-nodes in a network system. One popular network protocol is the Ethernet™ protocol.
The Ethernet protocol as defined by various standards, including the IEEE std. 802.3, published Mar. 8, 2002 is currently considered to be the dominant data networking technology. The Ethernet protocol is widely accepted as a means to communicate network packets to and from edge-devices (or end-stations) over a wider array of networks ranging from LANS to wireless networks. Ethernet has retained its viability as a communications technology even in the face of an explosive growth in demand for bandwidth and high data rate communications by evolving to meet extant network performance standards. While the original wirespeeds for Ethernet standards were only 10 megabits per second (Mbps), current rates are upwards of 40 gigabits per second (Gbps) with rates of 100 Gbps soon to be available. Ethernet® is a registered trademark of Xerox Corporation.
The Ethernet standard uses a packet (or frame) as a basic unit of data to communicate information over electrical, optical or wireless communications media to devices connected to the network. The Ethernet protocol specifies the rules (or standards) for constructing such packets or frames and this standardization has been one of the primary contributors to the success of the Ethernet protocol as a networking technology. The rules define, among other things, the minimum and maximum length of a packet and the information that must necessarily be present in each packet. A standard Ethernet packet is generally between 64 and 1518 bytes in length and includes 46 to 1500 bytes of data, plus a mandatory 18 bytes of header plus FCS information. Each packet (or frame) is required to include a source address (SA) and a destination address (DA) that uniquely identify the source and the recipient of the packet. In this regard, each network node (alternatively end-node, device, end-station, or edge-device) is associated with a unique Media Access Control (MAC) address that is 48 bits long.
Typically, the network comprises several local area networks (LANs), each of which may communicate through an intermediate network device, such as a bridge, switch or router, with the other LANs that comprise the network. The intermediate device does not originate traffic of its own although it may terminate (i.e. drop) packets that do not conform to the Ethernet standard (also known as “illegal packets”). Ethernet is a connectionless, broadcast based protocol intended for use in a shared medium. The intermediate devices use a forwarding table as well as the address information in the header of an incoming packet to decide where the incoming packet should be forwarded to. Forwarding tables can be permanently defined (static) or built by the intermediate device by learning the MAC addresses of devices on the LAN links. In this respect, every Ethernet compliant packet includes a mandatory 6-byte destination address (DA), a 6-byte source address (SA), two 2-byte E-type and a 4 byte FCS for a total of 20 bytes of packet overhead and 14 bytes of header overhead. A minimum Ethernet packet size of 64 bytes (not including the preamble) involves a header overhead of about (14/64)*100 or 22 percent. The size of the header affects the available bandwidth. It also affects the speed with which the header can be processed. Moreover, the Ethernet protocol imposes a fundamental limit on the global address space due to the finite number of address bits permitted in a Ethernet packet. The global address space limitation constrains the number of nodes in the network that can be explicitly addressed. Other nodes in the network may be considered to belong to a local network that is hierarchically below and extends from an explicitly addressed “root” node. The nodes in this local network may be addressed using logical addresses valid only in the local network. Computing and resolving such logical addresses imposes a computational overhead in addition to the header processing burden. Clearly, high-speed, Ethernet based communications in a local network may be problematic due to the latency introduced by header processing and bandwidth limits.
One of the approaches to address the aforementioned issues is the concept of a virtual local area network (VLAN). As opposed to a LAN, which represents a physical network system comprised of a plurality of network devices and the physical interconnection between them, such as for example, Ethernet™ or fiber optic links, a virtual local area network (VLAN) is logical segmentation of a single physical network into multiple networks. A virtual LAN (VLAN) is a collection of network nodes, perhaps on multiple physical LAN segments, that can communicate as if they were connected to the same physical LAN without the need to repetitively process the standard Ethernet header. A given set of network devices may logically belong to several VLANs each of which is capable of Ethernet based packet communication if the VLAN segmentation is done pursuant to the Institute of Electrical and Electronics Engineering (IEEE) 802.1Q draft standard Following the IEEE 802.1Q draft standard, the use of a Virtual LAN (VLAN) identification (ID) (VLAN ID) as prescribed in the Institute of Electrical and Electronics Engineering (IEEE) draft standard 802.1Q. VLANS are set up by inserting a tag, called a VLAN tag, into each Ethernet frame. The tagged frame carries VLAN membership information. The VLAN tag is 2 bytes in length. The last 12 bits of the tag is reserved for a VLAN identifier (VLAN ID). Tagging can be based on the source or destination Media Access Control (MAC) address. Each intermediate device maintains a table of MAC addresses and their corresponding VLAN memberships. VLAN-unaware devices transmit and receive normal MAC data frames, such as untagged data packets. VLAN-aware devices are capable of transmitting and receiving tagged data packets. The VLAN-aware devices switch packets based on the VLAN ID. The VLAN ID presents a smaller header processing overhead. This approach reduces the header processing overhead while a packet is being sent, but it does not change the overall length of the packets that must be communicated across the network. As noted above, an additional 2 bytes in the form of a 802.1q VLAN Tag added, are added to the header. For a minimum Ethernet packet size of 64 bytes (not including the preamble), the VLAN ID based approach introduces a header overhead of (20/64)*100 or about 31 percent.
It is well known that shorter packets dominate network traffic. In such instances, the information carried in the header is about the same as the actual data payload resulting in a significant header processing related overhead. Such a situation may arise, for example, where the packets represent small, periodic, distributed data transfers associated with cellular transmissions or Voice over Internet Protocol (VoIP) traffic. A prior art approach to improve the header overhead to data ratio is header compression. Header compression is typically used to reduce the header overhead when using protocols such as real time protocol (RTP), user datagram protocol (UDP) and Internet Protocol (IP) on slow and medium speed links such as, for example, an air link. Header compression involves the minimization of the bandwidth for information carried in headers by taking advantage of the fact that some fields in the headers of consecutive packets in a packet stream remain static or change in a predefined way. Currently, there are proposals to reduce the 40-byte RTP/UDP/IP header to 4-5 bytes for instance. An exemplary header compression scheme is described in S. Casner and V. Jacobson, “Compressing IP/UDP/RTP Headers for Low-Speed Serial Links,” IETF RFC 2508, the contents of which are hereby incorporated by reference. These header compression methods have to compensate for packet loss due to link errors and link latencies to prevent the de-compressor from appending incorrect header information to an out of sequence packet.
One attempt to improve the VLAN tag header has been described in U.S. Pat. No. 6,975,627 to Parry et al. which discloses a modification of tag fields in Ethernet data packets that relies on the existence, in current Ethernet standards, of a requirement for a header that precedes a tag that is used to denote a number identifying a virtual local area network. The invention is based on the use of the VLAN tag header to convey the selected or proprietary information where the VLAN identification field is modified by inserting in place of the VLAN tag header a field of the same size including selected information. The inserted field may include a first field indicating the presence of the VLAN identification field and a second field of selected information. The main purpose of the Perry invention is to modify a packet with selected information on the assumption that the egress port is proprietary, thus avoiding the difficulties of techniques that include adding data before or after existing data fields in the Ethernet protocol. Parry discloses the use of auto-negotiation to confirm that devices connected by a data link are compatible so as to ensure that the data link between the devices which are to form a common logical entity must be such that every packet intended for transmission by way of the link has a VLAN tag header. Parry's scheme allows control information to be passed between the units of the stack, or in general, different units within a physical system, with no increase in bandwidth for tagged packets. The stack is a plurality of devices, such as a multiplicity of hubs or switches, coupled together so that from the point of view of the external network, the plurality of coupled or “stacked” devices acts as a single logical entity. Parry's scheme inherits the 42 bytes/packet frame processing overhead associated with 802.1q Ethernet as previously described.
One area where the issue of header overhead takes on special importance is where Ethernet switching fabrics are utilized as backplane fabrics in a computing/communication system. US Publ. Appl. No. 20050091304, for example, discloses a control system for a telecommunication portal that includes a modular chassis having an Ethernet backplane and a platform management bus which houses at least one application module, at least one functional module, and a portal executive. In this patent application, a 1000 BaseT (Gigabit Ethernet) backplane provides a packet-switched network wherein each of the connected modules acts as an individual node on a network in contrast to a conventional parallel bus connection such as a PCI bus. US Publ. Appl. No. 20060123021 discloses a hierarchical packaging arrangement for electronic equipment that utilizes an Advanced Telecommunication Computing Architecture (ATCA®) arrangement of daughter boards in the for an Advanced Mezzanine Card (AMC™) that are interconnected with a hierarchical packet-based interconnection fabric such as Ethernet, RapidIO, PCI Express or Infiniband. RapidIO is a trademark of the RapidIO Trade Association. PCI Express is a trademark of the PCI-SIG. InfiniBand is a trademark of the IBTA (InfiniBand Trade Association). In this arrangement, the AMCs in each local cube are connected in a hierarchical configuration by a first, lower speed interface such a Gigabit Ethernet for connections within the local cube and by a second, higher speed interface such as 10 G Ethernet for connections among cubes. AdvancedTCA and the AdvancedTCA logo are registered trademarks of the PCI Industrial Computers Manufacturers Group. ATCA and the ATCA logo are trademarks of the PCI Industrial Computers Manufacturers Group. Other names and brands may be claimed as the property of others.
The problems of an Ethernet-switched backplane architectures in terms of latency, flow control, congestion management and quality of service are well known and described, for example, by Lee, “Computation and Communication Systems Need Advanced Switching,” Embedded Intel Solutions, Winter 2005. Intel is a registered trademark of Intel Corporation or its subsidiaries in the United States and other countries. These issues have generally discouraged the adoption of serial I/O protocols for communications between processors and memory that would typically be limited to the smaller physical dimensions of a circuit board or a computer or communication rack or cabinet having multiple cards/blades interconnected by a backplane. Instead, the trend has been to increase the capacity of individual chips and the size of each of the server blades in order to accommodate more processors and memory on a single chip or circuit board, thereby reducing the need for processor and memory interconnection over the backplane.
Another fundamental problem with Ethernet backplanes is that the standard Ethernet frames are less efficient than other packet based backplane technologies because of excessive packet (or frame) overhead which requires additional backplane bandwidth. For a line card to support a rate above 1 Gbit/s, would require the provisioning of a 10-Gbit/s Ethernet backplane link. In addition to the frame processing overhead, standard Ethernet lacks effective mechanisms for flow control, congestion management, and high availability. Although Ethernet provides three priority bits in the VLAN tag that can be used to provide such mechanisms, there is no industry standard on how to use these bits. Ethernet has no class based flow control mechanism, and only supports an XON/XOFF mechanism in some applications. Since most prior art systems utilize layer 2 Ethernet switching which does not have these QoS or HA features, these prior art systems are unlikely candidates for solutions that would overcome the limitations of the Ethernet backplanes.
Another downside of using Ethernet as a backplane switching fabric for silicon-to-silicon interconnect is the lack of an infrastructure to handle congestion management. This aspect is described in Robert Brunner, Shashank Merchant, “Congestion Management Requirements and Proposals—A TEM's View”, the contents of which are incorporated hereby in there entirety. The congestion may arise from, for example, an unavoidable rate mismatch between the blades and between multiple chasses. Unfortunately, unlike communications over a computer network where some latency is tolerable, intra-device and inter-blade communications cannot tolerate packets being discarded in the switching fabric.
While many of the above mentioned limitations could be overcome by employing new mechanisms that utilize non-Ethernet components, such an approach cannot avoid the penalty of losing the cost advantages gained through Ethernet's economies of scale. An exemplary non-standard mechanism would entail appending data onto the beginning or end of an existing packet's data field. Recall that standard Ethernet packets have a prescribed maximum length. Appending data to an Ethernet packet can result in an illegal packet, i.e. one whose length exceeds the maximum packet length. An illegal packet will likely be ignored or dropped by intermediate devices as the packet progresses thorough the network.
In view of the above, it would be advantageous to provide an enhanced Ethernet protocol that could overcomes the shortcomings of the existing approaches to improving header efficiency and, particularly, that could improve over the prior art Ethernet switching backplanes in the area of header overhead, link utilization, quality of service (QoS), high availability (HA), and latency while being inherently secure from out-of-the-system snooping.